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Abstract:  Injuries  are  the  leading  cause  of  morbidity  and  mortality  confronting  U.S.  military  forces 
in  peacetime  or  combat  operations.  Not  only  are  injuries  the  biggest  health  problem  of  the  military 
services,  they  are  also  a  complex  problem.  The  leading  causes  of  deaths  are  different  from  those  that 
result  in  hospitalization,  which  are  different  from  those  that  result  in  outpatient  care.  As  a  conse¬ 
quence,  it  is  not  possible  to  focus  on  just  one  level  of  injury  severity  if  the  impact  of  injuries  on 
military  personnel  is  to  be  reduced.  To  effectively  reduce  the  impact  of  a  problem  as  big  and  complex 
as  injuries  requires  a  systematic  approach. 

The  purpose  of  this  paper  is  to:  (1)  review  the  steps  of  the  public  health  process  for  injury 
prevention;  (2)  review  literature  on  evaluation  of  the  scientific  quality  and  consistency  of  information 
needed  to  make  decisions  about  prevention  policies,  programs,  and  interventions;  and  (3)  summa¬ 
rize  criteria  for  setting  objective  injury  prevention  priorities.  The  review  of  these  topics  will  serve  as 
a  foundation  for  making  recommendations  to  enhance  the  effectiveness  of  injury  prevention  efforts 
in  the  military  and  similarly  large  communities.  This  paper  also  serves  as  an  introduction  to  the  other 
articles  in  this  supplement  to  the  American  Journal  of  Preventive  Medicine  that  illustrate  the  recom¬ 
mended  systematic  approach. 

(Am  J  Prev  Med  2010;38(1S):S1-S10)  Published  by  Elsevier  Inc.  on  behalf  of  American  Journal  of  Preventive 
Medicine 


Introduction 

Injuries  in  the  Military:  a  Large  Problem 

Injuries  are  the  biggest  health  problem  confronting 
U.S.  military  forces  in  peacetime  and  combat  opera¬ 
tions.^’^  Injuries  result  in  over  1.8  million  medical 
encounters  annually  across  the  military  services  and  af¬ 
fect  more  than  800,000  individual  service  members.^  The 
second  leading  cause  of  medical  encounters,  mental  dis¬ 
orders,  results  in  about  750,000  encounters  annually,  af¬ 
fecting  about  190,000  service  members.  Historically,  in¬ 
juries  have  been  shown  to  be  the  leading  cause  of  deaths, 
disabilities,  hospitalizations,  and  outpatient  visits. 
While  battle  injuries  are  the  leading  cause  of  death  in 
Operations  Iraqi  Freedom  (OIF)  and  Enduring  Freedom 
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(OEF),  nonbattle  injuries  are  the  leading  cause  of  health 
conditions  serious  enough  to  require  aero-medical 
evacuations  out  of  the  theater  of  operations.  Nonbattle 
injuries  account  for  about  35%  of  such  medical  evacu¬ 
ations,  compared  to  16%  for  battle  injuries  and  7%  for 
digestive  diseases,  the  leading  non-injury  reason  for 
medical  evacuation  (Keith  Hauret,  U.S.  Army  Center 
for  Health  Promotion  and  Preventive  Medicine,  un¬ 
published  data,  2008).  Relative  to  other  health  prob¬ 
lems,  injuries  have  the  biggest  impact  on  the  health  and 
combat  readiness  of  military  personnel. 

In  the  past,  military  surveillance  of  injuries  and 
“accidents”  has  focused  primarily  on  fatalities,  espe¬ 
cially  motor  vehicle  and  aviation  fatalities.  Since  the 
late  1990s,  however,  increasing  attention  has  been  di¬ 
rected  toward  nonfatal  injuries  following  establish¬ 
ment  of  the  Defense  Medical  Surveillance  System  in 
1997.®  As  a  result  of  the  recent  focus  on  nonfatal  inju¬ 
ries,  it  has  been  shown  that  for  every  noncombat  injury 
death  of  a  military  service  member  there  are  33  hospi¬ 
talizations  and  3800  outpatient  clinic  visits  for  inju¬ 
ries.®  It  has  also  been  estimated  that  injuries  result  in 
about  25,000,000  days  of  limited  duty  among  service 
members  annually.®  It  is  clear  that  injuries  are  a  tre- 
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mendous  drain  on  military  manpower  during  peace¬ 
time  and  times  of  armed  conflict. 

Injuries  in  the  Military:  a  Complex  Problem 

In  addition  to  being  a  large  problem,  injuries  are  also  a 
complex  problem  for  the  military.  Among  other  com¬ 
plexities,  as  with  civilian  communities,  the  leading  causes 
of  injury  vary  widely  depending  on  the  level  of  severity  of 
injuries.  For  instance,  historically,  motor  vehicle  crashes 
have  been  the  leading  cause  of  unintentional,  nonbattle 
injury  deaths  across  all  the  military  services,  accounting 
for  55%  to  64%  of  all  unintentional  injury  deaths,^®  re¬ 
sulting  in  five  to  ten  times  as  many  deaths  as  the  next 
leading  specific  injury  causes  (drowning,  fires/burns,  or 
falls,  depending  on  the  service).^  On  the  other  hand,  the 
top  three  causes  of  injury  hospitalization  of  military  per¬ 
sonnel  have  been  documented  to  be  falls,  athletics 
(sports),  and  motor  vehicle  crashes.®  In  2006,  the  leading 
cause  of  injury  hospitalizations  for  military  personnel 
was  falls  and  near  falls  (slips  and  trips),  which  accounted 
for  17.5%  of  such  hospitalizations,  followed  by  motor 
vehicle  mishaps  at  15.4%,  and  sports  and  athletics  at 
13.1%  (Michelle  Canham-Chervak,  U.S.  Army  Center  for 
Health  Promotion  and  Preventive  Medicine,  unpub¬ 
lished  data,  2007). 

The  same  three  causes  of  injuries — falls,  sports,  and  mo¬ 
tor  vehicle  mishaps — are  also  the  leading  causes  of  medical 
evacuations  from  operations  in  Iraq  and  Afghanistan.^  The 
only  data  readily  available  on  causes  of  outpatient-treated 
injuries  come  from  army  field  investigations.  That  data 
indicated  that  physical  training  (i.e.,  exercise)  is  the  lead¬ 
ing  cause  of  outpatient  injury  visits,  accounting  for  25% 
to  40%  of  such  injuries. On  the  other  hand,  motor 
vehicle  mishaps  account  for  less  than  5%  of  all  injuries 
treated  in  outpatient  clinics  and  rank  no  higher  than  7th 
or  8th  when  compared  to  other  causes.  It  is  clear  from 
data  such  as  these  that  if  priorities  for  military  injury 
prevention  were  based  on  fatalities,  the  major  causes  of 
the  majority  of  injuries,  physical  training  and  falls,  would 
not  be  addressed. 

Adding  further  to  the  complexity  of  the  problem  of 
injuries,  the  circumstances  of  injuries  resulting  from  sim¬ 
ilar  causes  can  be  quite  different.  As  an  example,  falls  can 
occur  from  stairs,  ladders,  and  other  heights,  and  on  level 
surfaces  during  garrison  or  combat  conditions. Like¬ 
wise,  athletics  result  in  frequent,  sometimes  serious,  inju¬ 
ries  associated  with  a  variety  of  sports  occurring  under 
varied  circumstances.^’ 

With  regard  to  prevention,  success  has  been  achieved 
where  attention  has  been  focused  and  surveillance  sys¬ 
tems  are  in  place.  For  example,  just  as  in  the  civilian 
community,  the  military  has  had  great  success  preventing 
injuries  and  deaths  associated  with  privately  owned  mo¬ 


tor  vehicle  crashes.^®  Furthermore,  because  an  abundance 
of  evidence  for  further  prevention  of  motor -vehicle  crashes 
is  available  from  established  government  organizations  such 
as  the  National  Highway  Traffic  Safety  Administration,  the 
CDC,  the  Insurance  Institute  for  Highway  Safety,  and 
academic  organizations,  the  military  can  adopt  civilian 
approaches  already  demonstrated  to  be  effective.^®  But 
for  most  injury  problems,  even  large,  potentially  serious 
injury  problems  such  as  falls,  very  little  prevention  infor¬ 
mation  is  available.  Research  needs  to  be  conducted  be¬ 
fore  policies  and  programs  are  implemented  when  sys¬ 
tematic  reviews  determine  that  scientific  information  is 
scant  and  where  gaps  in  knowledge  about  prevention 
exist. 

Purpose 

As  with  any  large  community  or  occupational  group,  for 
a  public  health  problem  as  big  and  complex  as  injuries  in 
the  military,  a  systematic  approach  to  planning  and  set¬ 
ting  priorities  is  needed  for  prevention  activities  to  suc¬ 
ceed.  Because  resources  for  prevention  are  scarce,  a  pro¬ 
cess  is  needed  for  setting  priorities  that  identifies  not  only 
effective  countermeasures,  but  also  those  strategies  that 
affect  the  health  of  the  largest  number  of  personnel  at  the 
lowest  costs. When  scientific  evidence  is  available,  a 
process  for  evaluating  the  quality  of  individual  studies 
and  a  mechanism  for  making  recommendations  for  the 
aggregate  findings  on  a  particular  injury  problem  are 
needed.  The  purpose  of  this  paper  is  to  outline  a  system¬ 
atic  process  for  identifying  the  largest,  most  severe  injury 
problems  for  which  effective  prevention  strategies  have 
been  developed,  and  targeting  those  problems  for  inter¬ 
vention.  An  evidence-based  approach  to  identifying 
problems  for  which  effective  prevention  strategies  exist  is 
described  and  used  as  a  foundation  for  making  recom¬ 
mendations  that  could  be  applied  to  the  military  or  other 
similarly  large  populations. 

A  Systematic  Process  for  Injury  Prevention 

The  public  health  approach.  Modern  epidemiology 
has  shown  that  injuries  are  not  “accidents”;  they  are  pre¬ 
dictable  and  preventable.  A  comprehensive  public  health 
approach  for  the  prevention  of  injuries  has  previously 
been  recommended  for  the  military  services. The 
primary  recommendation  entailed  establishing  the  five 
functional  elements  of  the  public  health  approach  for 
inj  ury  prevention  listed  in  T able  1 .  For  a  large  community 
or  organization  such  as  the  military  to  successfully  pre¬ 
vent  injuries,  it  is  necessary  for  each  of  the  five  functional 
elements  to  be  operating.  Although  the  approach  does 
not  necessarily  need  to  be  carried  out  in  sequential  order, 
all  of  the  steps  are  necessary  in  order  to  successfully 
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Table  1.  Functional  steps  of  the  public  health  approach  to  Injury  prevention® 


Functional  step  of 
prevention  process 

Description  of  function 

1.  Surveillance 

Medical  and  safety  surveillance  routinely  tracks  frequencies,  rates,  and  trends  in  injuries  and 
other  health  problems.  The  data  are  used  to  identify  ongoing  and  emergent  problems  and 
to  help  set  priorities.  Surveillance  can  also  help  monitor  prevention  policy  and  program 
effectiveness. 

2.  Research  and  field 
investigations  on  risk 
factors  and  causes 

Research  and  to  some  extent  public  health  field  investigations  provide  information  on  the 
incidence  of  injuries  and  other  health  problems  and  determine  causes  and  risk  factors  for 
health  problems. 

3.  Research  on  interventions 

Research  may  also  entail  conducting  intervention  trials,  both  randomized  and  nonrandomized, 
to  determine  what  works  to  prevent  injuries  and  other  health  problems.  Intervention  trials 
provide  information  on  the  efficacy  of  prevention  strategies. 

4.  Program  and  policy 
implementation 

Policymakers,  worksite  supen/isors,  military  commanders,  and  other  authorities  direct 
implementation  of  injury  prevention  and  other  public  health  policies,  programs,  and 
strategies  to  protect  populations  and  communities. 

5.  Evaluation  and  monitoring 
of  programs  and  policies 

Once  policies,  programs,  and  strategies  are  implemented  the  effectiveness  of  those  activities 
should  be  evaluated  to  determine  the  effectiveness  of  the  actions.  Surveillance  data  can 
also  be  used  to  monitor  ongoing  effectiveness. 

'^Adapted  from  other  published  reports. The  public  health  approach  was  first  described  by  Mercy  et  al.^® 


prevent  injuries  over  time.  Great  strides  have  been  made 
since  the  initial  recommendation  of  the  five -step  public 
health  approach  to  the  Armed  Forces  Epidemiological 
Board  in  1996.^°  Routine  medical  surveillance  of  injuries 
resulting  in  hospitalization  of  military  service  members 
and  also  those  treated  in  outpatient  clinics  has  been  im¬ 
plemented.  Additionally,  the  means  to  evaluate  pubhc 
health  practices  implemented  to  prevent  injuries  has  been 
demonstrated.^^’^^  The  steps  for  which  the  least  progress  has 
been  made  are  research.  While  occasional  ad  hoc  injury 
research  initiatives  arise,  at  this  time  there  is  no  dedicated 
injury  prevention  research  objective  or  program  for  the  mil¬ 
itary.  In  addition,  injury  cause  coding  of  outpatient  data  is 
needed  to  prioritize  and  guide  prevention  efforts.  Despite 
great  progress,  for  injury  prevention  in  the  military  to  be 
effective,  all  of  the  steps  of  the  process  need  to  be  improved 
for  each  of  the  services. 

The  evidence-based  process.  In  addition  to  the  five 
steps  of  the  public  health  approach,  cost-effective  injury 
and  disease  prevention  requires  an  evidence -based  mech¬ 
anism  for  prioritizing  prevention  activities  and  allocation 
of  public  health  and  prevention  resources  (Table  2).  Pri¬ 
ority  should  be  given  to  problems  for  which  there  is 
scientific  evidence  of  effective  prevention  policies,  pro¬ 
grams,  or  interventions. Great  progress  has  been 
made  in  evidence-based  decision  making  in  preventive 
medicine  and  public  health  over  the  last  20  years  in  the 
U.S.,  starting  with  the  U.S.  Preventive  Services  TaskForce 
(USPSTF)  in  the  late  1 980s. As  described  by  Briss  and 
McGinnis,  the  USPSTF  first  applied  the  evidence-based 
process  to  the  evaluation  of  clinical  preventive  services  in 


the  Guide  to  Glinical  Preventive  Services  in  1989.^®’^® 
With  the  development  of  the  Guide  to  Gommunity  Pre¬ 
ventive  Services  in  2000,  that  process  was  extended  to 
community  public  health. The  process  for  identify¬ 
ing  successful  evidence-based  prevention  strategies  and 
setting  public  health  and  safety  priorities  has  gained 
enough  credence  to  recommend  wide  implementation. 

The  key  step  of  the  evidence-based  process  is  the  eval¬ 
uation  of  the  quality  of  individual  prevention  studies  or 
investigations  (Table  2).®^’®^  The  next  step  of  the  process 
is  development  of  recommendations  for  prevention 
based  on  the  overall  quality  and  consistency  of  the  evi¬ 
dence  supporting  or  refuting  the  effectiveness  of  preven¬ 
tion  strategies. Earlier  steps  of  the  evidence-based 
decision-making  process  begin  with  defining  the  prob¬ 
lem  and  systematic  reviews  of  the  scientific  literature,  or 
meta-analyses.  The  final  steps  are  to  set  priorities  for 
prevention  and  research  based  on  the  magnitude  or  se¬ 
verity  of  public  health  problems;  the  strength  and  consis¬ 
tency  of  the  evidence  that  effective  solutions  for  the  prob¬ 
lems  exist;  and  a  determination  that  the  identified 
prevention  strategies  are  feasible  to  implement. 

Surveillance  and  the  evidence-based  process.  The 

evidence-based  process  begins  with  identification  of  the 
biggest  and  most  severe  health  problems  affecting  a  com¬ 
munity  or  population. Health  and  safety  surveillance 
and  surveys  are  the  logical  means  for  identifying  the  most 
common  and  severe  injury  and  other  health  problems  of 
a  community.  In  the  past,  the  public  health  importance  of 
injuries  and  other  health  problems  has  been  established 
primarily  using  fatality  data.®®  In  the  military  services. 
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Table  2.  Steps  of  evidence-based  public  health  decision-making  process 


Step  of  process 

Description  of  step 

1.  Identification  of  biggest  or 
most  severe  probiems 

The  first  step  of  the  evidence-based  public  health  process  utilizes  medical,  safety,  and  other 
sun/elllance  and  survey  data  sources  to  Identify  causes  or  types  of  injury  with  high  rates  or 
indicators  of  severity  to  target  for  potential  prevention. 

2.  Search  for  evidence  of 
effective  prevention 

The  second  step  of  the  process  uses  knowledge  of  the  most  significant  injury  problems 
confronting  a  population  from  Step  1  to  focus  systematic  reviews  of  the  scientific  literature 
on  those  problems  to  determine  what  evidence  exists  for  their  prevention. 

3.  Evaiuation  of  quaiity  of 
evidence  for  prevention 

The  third  step  of  the  process  evaluates  the  quality  of  individual  research  studies  using 
predetermined  criteria  to  assess  strengths  and  weaknesses  of  design,  execution,  and 
analysis. 

4.  Recommendations  based  on 
strength  and  consistency  of 
evidence 

The  fourth  step  of  the  process  assesses  the  strength  and  consistency  of  the  overall 
evidence  that  interventions  work  to  prevent  the  problems  identified  as  a  foundation  for 
recommendations.  Note:  No  one  study  design  addresses  all  the  questions  requiring 
answers  about  effectiveness,  harms,  and  real-world  feasibility.  One  study  is  not  sufficient 
to  make  evidence-based  recommendations. 

5.  Prioritization  of 
interventions 

The  fifth  step  applies  predetermined  criteria  to  rank  prevention  strategies  for  allocation  of 
resources  and  implementation  based  on  the  magnitude  or  severity  of  a  problem,  its 
preventability  (evidence  of  effective  interventions),  and  the  feasibility  of  implementation. 

6.  Identification  of  research 
gaps 

The  sixth  and  final  step  of  the  evidence-based  prevention  process  can  take  place 
concurrently  with  the  fifth.  This  step  identifies  gaps  in  knowledge  of  what  prevents  the 
most  significant  health  problems  confronting  a  population  and  targets  them  for  more 
research. 

top  priorities  for  injury  prevention  are  still  predicated  on 
the  leading  causes  of  deaths — motor  vehicle  and  aviation 
mishaps.  The  prioritization  process  should  include  mea¬ 
sures  not  just  of  mortality,  but  also  morbidity  measures 
such  as  disabilities,  hospital  discharges,  and  visits  for 
emergency  and  other  outpatient  treatment,  among  oth¬ 
ers. Using  fatality  data  for  setting  injury  prevention 
priorities  can  be  particularly  misleading  as  the  most  fre¬ 
quent  causes  of  injuries  do  not  necessarily  result  in 
death. Also,  as  highlighted  earlier,  because  the  leading 
causes  of  injury  deaths  are  different  than  the  leading 
causes  of  the  more  numerous  nonfatal  injuries,  reducing 
the  leading  causes  of  deaths  may  have  little  impact  on  the 
overall  burden  of  injuries  on  a  population.  As  a  conse¬ 
quence,  in  the  initial  phase  of  identifying  the  most  impor¬ 
tant  injury  problems  of  a  community,  both  magnitude 
and  severity  of  injuries  should  be  considered  using  fatal¬ 
ity,  disability,  hospitalization,  and  outpatient  data. 

Systematic  reviews  and  the  quaiity  of  evidence.  In 

targeting  and  conducting  effective  injury  prevention,  it  is 
not  enough  to  know  what  the  biggest  injury  problems  are. 
It  is  also  necessary  to  know  which  are  preventable.  A 
process  for  identifying  and  evaluating  the  evidence  for 
what  works  to  prevent  injuries  is  also  essential  (Table  2). 
A  number  of  approaches  have  been  established  for  eval¬ 
uating  the  effectiveness  of  interventions  to  treat  or  pre¬ 
vent  health  problems.  The  best  known  is  the  process 
established  by  the  U.S.  Preventive  Services  Task  Force, 
which  has  been  well-described  elsewhere.“’^®’'^°  Similar 


processes  have  been  adopted  by  other  groups  and  organi¬ 
zations. Most  recently,  a  similar  process  has  been 
adopted  by  the  Guide  for  Community  Preventive  Ser¬ 
vices. What  the  Guide  for  Community  Preven¬ 
tive  Services  and  other  such  evidence-based  processes 
have  in  common  is  the  initiation  of  the  process  with  a 
systematic  review  of  the  literature  using  a  well-defined, 
pre-established  approach.  Once  the  most  prevalent 
and  severe  public  health  problems  have  been  identi¬ 
fied,  the  second  step  serves  to  identify  potential  inter¬ 
ventions/countermeasures  that  have  been  scientifically 
evaluated  and  found  effective. 

After  the  literature  searches  have  been  completed,  the 
next  step  is  to  assess  the  quality  of  the  science  for  identi¬ 
fied  studies  and  to  characterize  the  health  outcomes  asso¬ 
ciated  with  the  interventions  studied  and  the  size  of  the 
effects  due  to  the  interventions.  There  is  a  growing  con¬ 
sensus  that,  whether  one  is  assessing  the  effectiveness  of  a 
medication,  clinical  preventive  services,  or  a  community 
preventive  service,  more  than  just  the  beneficial  effect  of 
an  intervention  must  be  considered  when  making  recom¬ 
mendations  for  prevention.  The  potential  harms  of  an 
intervention  must  be  assessed  To  work  weU, 

the  process  must  have  (1)  a  standardized  method  of  find¬ 
ing  evidence  to  assess,  (2)  a  standard  set  of  considerations 
in  evaluating  the  quality  of  individual  scientific  studies, 
and  (3)  a  method  of  arriving  at  a  composite  score  for  each 
study  on  a  particular  prevention  strategy  that  can  be 
compared  to  other  studies.  Such  systematic  reviews  (lit- 
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Table  3.  Levels  of  recommendations  for  injury  prevention  strategies® 


Recommendation 

Reasons  for  recommendation 

Strongly  recommend 

Good  data  on  effectiveness  exist,  some  of  it  high  quality,  and  findings  across  studies  are  consistent. 

Effect  sizes  are  substantial. 

Recommend 

At  least  fair  evidence  of  inten/ention  effectiveness  exists,  and  findings  of  effectiveness  are  mostly 
consistent.  Effect  sizes  may  be  modest. 

No  recommendation 

Benefits  and  harms  too  close  to  make  a  recommendation. 

Recommend  against 
use 

Data  from  studies  of  adequate  sample  size  to  show  intervention  effects  of  modest  magnitude  do  not 
indicate  that  the  intervention  is  effective  or  that  the  harms  of  the  intervention  outweigh  the  benefits. 

Insufficient  evidence 

Insufficiency  of  evidence  may  result  from  a  complete  lack  of  data,  few  studies,  or  inconsistency  of 
results. 

Expert  opinion 

In  the  absence  of  scientific  evidence  on  the  effectiveness  of  inten/entions  (e.g.,  insufficient  evidence), 
utilization  of  expert  opinion  or  consensus  opinions  on  recommendations  for  prevention  may  be 
warranted  for  urgent  health  problems.  When  the  basis  for  an  intervention  is  expert  opinion,  the 
intervention  implemented  should  be  rigorously  evaluated  and  closely  monitored  for  effectiveness. 

'^Adapted  from  other  published  reports. 


erature  reviews  coupled  with  quality  assessments)  are 
now  viewed  as  a  critical  part  of  the  public  health  decision- 
making  process. 

Completing  the  literature  search  and  evaluation  pro¬ 
cess  is  time  consuming  and  rigorous.  As  a  consequence, 
Harris  et  al.,^^  in  writing  about  the  USPSTF,  state  that 
“limited  resources  and  time  requires  compromises  in 
the  intensity  of  reviews  ....  One  strategy  is  .  .  .  topic 
prioritization ....  Another  strategy ...  is  to  focus  the  review 
on  the  questions  and  evidence  most  critical  to  making  rec¬ 
ommendations.”  This  type  of  process  has  been  applied  to 
setting  priorities  for  military  injury  prevention,  and  an  ex¬ 
pedited  process  for  more  rapid  evaluation  by  public  health 
and  safety  organizations  has  been  recommended.®’^^  To 
facilitate  more  rapid  transmission  of  evidence-based  in¬ 
jury  prevention  information  to  decisionmakers  and  poli¬ 
cymakers,  an  expert  military  panel  established  an  expe¬ 
dited  process  for  scientific  study  evaluation. Thus 
several  approaches  can  be  used  to  facilitate  more  rapid 
transmission  of  information  from  evaluators  to  decision 
makers,  including  focusing  the  systematic  review  process 
on  the  most  important  interventions  and  expediting  the 
review  process  itself 

Systematic  reviews  and  the  strength  and  consistency 
of  evidence.  Following  the  identification  of  evidence 
sources  (studies)  and  the  evaluation  of  the  quality  of 
individual  studies,  the  next  step  is  translation  of  the  body 
of  the  evidence  as  a  whole  into  recommendations.  This 
step  of  the  evidence-based  process  entails  the  assessment 
of  the  overall  strength  and  consistency  of  the  evidence  for 
a  particular  intervention. As  mentioned  earlier, 
not  only  must  the  process  evaluate  how  effective  an 
intervention  strategy  is  at  preventing  injuries  or  other 
health  outcomes  but  also  if  any  harms  might  arise  from 


implementation. In  making  recommen¬ 
dations,  information  on  the  quality  and  consistency  of 
evidence  that  a  strategy  works  must  be  balanced 
against  potential  harms  and  the  costs  implementation 
may  impose. 

In  addition  to  weighing  effectiveness,  harms,  and  costs 
in  the  process  of  making  public  health  recommendations, 
consideration  should  be  given  to  what  needs  to  be  done  in 
situations  when  an  urgent  public  health  or  safety  problem 
exists,  but  there  is  insufficient  scientific  evidence  of  inter¬ 
ventions  that  work  to  prevent  the  problem. In  a  rating 
scheme  for  recommendations  by  the  Strength  of  Recom¬ 
mendation  Taxonomy  (SORT),  EbeU^^  indicates  that  the 
lowest  level  of  evidence  is  consensus  or  usual  practices 
(i.e.,  expert  opinion).  In  his  discussion  of  the  USPSTF, 
Harris  warns  that  if  evidence  is  deemed  insufficient  to 
make  a  recommendation  to  provide  preventive  services, 
then  decision  makers  must  rely  on  factors  other  than 
science. Claxton"^^  addresses  the  issue  directly,  stating 
that  a  method  is  needed  for  acquiring  “judgments  from 
experts  when  no  evidence  is  available.” 

With  the  above  considerations  in  mind,  a  set  of  ratings 
for  recommendations  has  been  made  that  is  suitable  for 
use  in  a  large  population  that  frequently  confronts  new 
and  significant  public  health  and  safety  problems,  many 
of  which  may  be  of  an  urgent  nature  (Table  3).  The 
proposed  categories  of  recommendations  are  as  follows: 
strongly  recommend,  recommend,  no  recommendation, 
insufficient  evidence,  and  expert  opinion.  To  accommo¬ 
date  the  inevitable  situation  in  which  the  military  and 
other  organizations  need  recommendations  for  interven¬ 
tion  when  no  evidence  of  preventability  exists,  an  addi¬ 
tional  category  for  expert  opinion  was  added.  It  should  be 
noted,  however,  that  when  interventions  without  clear 
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evidence  of  effectiveness  are  implemented,  they  should  be 
carefully  evaluated. 

Study  design  and  trade-offs  in  vaiidity  of  evidence.  An 

issue  of  importance  to  the  process  of  evaluating  the 
quality  and  strength  of  evidence  supporting  preven¬ 
tion  is  that  of  study  type  or  design.  In  the  past,  the  only 
acceptable  standard  was  an  RCT.  There  is,  however,  a 
growing  consensus  that  RCTs  are  not  necessarily  the 
only  acceptable  evidence  or  even  the  gold  standard, 
especially  for  nonpharmaceutical,  nonclinical,  community- 
based  interventions.^®’^^’'^'’’'^^’"'^’'*®  Even  the  USPSTF  accepts 
study  types  other  than  RCTs  as  evidence. This  consen¬ 
sus  has  arisen  from  the  growing  awareness  of  the  short¬ 
comings  of  RCTs  in  documenting  the  harms  or  adverse 
outcomes  of  interventions,  the  inability  to  provide  an 
accurate  assessment  of  the  magnitude  of  health  benefits 
of  an  intervention  in  a  non-experimental  setting,  and 
impracticality  of  conducting  randomized  studies  in  many 
circumstances.  While  RCTs  may  have  greater  internal 
validity,  they  lack  the  external  validity  offered  by  other 
study  designs.  Atkins  et  al."‘®  specifically  note  that  RCTs 
“may  not  give  an  accurate  picture  of  the  impact  of  a  policy 
decision  under  real-world  conditions.” 

Because  of  time  and  funding  constraints,  RCTs  fre¬ 
quently  employ  intermediate  outcomes.  This  is  not  accept¬ 
able.  Several  authors  caution  that,  in  determining  the  effec¬ 
tiveness  of  an  intervention,  it  is  essential  that  the  health 
outcomes  of  interest  be  assessed,  unless  the  link  between 
an  intermediate  outcome  and  the  occurrence  of  the  health 
outcome  of  interest  is  well  established,  such  as  the  link  be¬ 
tween  seat  belt  use  and  decreased  risk  of  traffic  fatalities. 

A  variety  of  valid  alternative,  nonrandomized  study  de¬ 
signs  maybe  useful  in  assessing  the  effectiveness  of  interven¬ 
tions  intended  to  be  implemented  on  a  wide  scale  in  com¬ 
munities  and  populations.  These  study  types  include 
nonrandomized  prospective  and  retrospective  cohort  stud¬ 
ies,  pre-post  studies,  time-series,  case-  control,  and  natural 
experiments,  as  well  as  other  quasi-experimental  or  obser¬ 
vational  types  of  studies.^®’"'^’'*®  Such  studies  may  be  better  at 
determining  the  efficacy  of  interventions  in  real-world  com¬ 
munity  settings  than  RCTs.  Teutsch  notes  that  data  from 
RCTs  is  scarce  for  many  interventions,  so  investigators 
should  not  be  deterred  from  using  other  more  practical 
study  designs.^®  Group  randomized  trials  provide  another 
alternative  to  RCTs.'‘^’"‘®  What  is  important  in  choosing  a 
study  design  is  that  it  be  able  to  determine  whether  imple¬ 
mentation  of  an  intervention  changed  the  incidence  of  the 
health  outcome  of  interest. 

Furthermore,  Mercer  et  al."^^  state  “No  one  study  estab¬ 
lishes  causality.”  Rather,  consistent  outcomes  from  mul¬ 
tiple  studies  make  a  better  foundation  for  evidence-based 
health  and  public  health  policy.  Atkins  et  al."^®  caution 


that  “policymakers  should  be  skeptical  of  evidence  de¬ 
rived  from  a  single  study.”  The  fact  that  no  one  study  or 
study  type  is  an  adequate  basis  for  policy  and  public 
health  decisions  argues  for  greater  use  of  systematic  re¬ 
views  that  make  use  of  aU  available  evidence  regardless  of 
study  type,  published  or  unpublished.'*®  Weighing  ev¬ 
idence  from  multiple  studies  of  different  types  provides  a 
greater  opportunity  to  balance  effectiveness  against  harms 
and  costs.  While  RCTs  provide  confidence  that  study  find¬ 
ings  are  not  due  to  chance  or  bias  resulting  from  inadequate 
study  design  (i.e.,  internal  validity),  other  types  of  well-de¬ 
signed  studies  may  be  more  practical,  less  costly,  and  more 
generalizable  (i.e.,  externally  valid). 

Criteria  for  setting  priorities.  The  need  for  a  mecha¬ 
nism  for  setting  priorities  for  allocation  of  resources  for 
prevention  of  injuries  and  disease  is  widely  recog- 
nized.*^-^*-^®-"®’''''-**-*®-^  The  Institute  of  Medicine’s 
Committee  on  Injury  Prevention  and  Control  stated  that, 
for  injury  prevention,  “Whatever  the  overall  level  of  pub¬ 
lic  investment . . .  priorities  for  research  and  social  action 
must  be  set.  The  challenge  facing  the  field  is  developing 
criteria  for  setting  these  priorities.”®® 

A  number  of  approaches  to  establishing  prevention  pri¬ 
orities  employing  a  variety  of  criteria  have  been  suggested, 
including  using  the  burden  of  disease,®^  consideration  of  the 
magnitude,  severity,  and  costs  of  problems*®’®®  or  these  in 
combination  with  preventability/effectiveness*^  and  feasi¬ 
bility  factors  such  as  acceptability,  available  resources,  and 
legal  authority. ^*’^®’®‘’’®'*  Runyan*®’^®  and  Fowler^*  have  de¬ 
scribed  more  comprehensive  criteria  than  others  and  cre¬ 
ated  decision  matrices  specifically  for  setting  injury  preven¬ 
tion  priorities.  Their  criteria  for  setting  injury  prevention 
priorities  can  be  aggregated  into  several  overarching  catego¬ 
ries  including  effectiveness  (preventability  considering  ben¬ 
efits  and  harms),  costs,  feasibility  (funding,  infrastructure, 
personnel,  legal  authority),  acceptability  (social  and  politi¬ 
cal),  and  sustainability. 

Process  for  setting  priorities  based  on  criteria.  Once 
criteria  are  established  for  setting  priorities,  a  process 
for  ranking  the  potential  priorities  must  be  developed. 
Fowler®*  described  a  qualitative  process  using  rankings 
of  “high,”  “moderate,”  and  “low”  for  each  criterion  in  a 
decision  matrix.  Runyan®®  suggested  that  either  qualita¬ 
tive  or  quantitative  methods  for  applying  criteria  can  be 
employed  to  set  priorities,  provided  that  decision  makers 
consider  the  most  important  factors  likely  to  determine 
policy,  program,  or  intervention  success.  Claxton  et  al.** 
expressed  a  preference  for  a  quantitative  approach  to 
such  decision  making,  stating:  “In  particular,  evidentiary 
criteria  are  not  tied  formally  and  quantitatively  to  bene¬ 
fits,  risks,  and  costs  associated  with  an  intervention  and  as 
a  result  do  not  maximize  health  benefits.”  As  a  conse- 
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Table  4.  Priority-setting  criteria  employed  by  military  injury  prevention  working  groups 


Criterion 

Scoring 

A.  Program  or  policy  is  consistent  with  mission  of  the  working  group/organization 

If  YES — Continue  with  scoring. 

If  NO — Stop  here. 

B.  Importance  of  problem  to  public  health  or  workplace  productivity 

(10  points;  l=low,  10=high) 

Considerations: 

1.  Magnitude  of  the  probiem  (e.g.  frequency,  incidence) 

2.  Severity  of  probiem  (e.g.,  degree  of  effect  on  personnei  health  and  performance) 

3.  Cost  of  the  problem  (e.g.,  training,  property,  and  personnel  costs) 

4.  Size  of  population  at  risk 

5.  Degree  of  concern  (e.g.,  leadership  concern,  public  and  Service  member  concern. 

visibility  of  problem) 

C.  Preventability  of  problem 

(10  points;  l=low,  10=high) 

Considerations: 

1.  Cause(s)  are  identifiable 

2.  Risk  factors  are  modifiable 

3.  Proven  prevention  strategies  that  reduce  existing  injury  rates  exist 

4.  Prevention  strategies  that  reduce  existing  injury  rates  can  be  designed 

5.  Effect  size 

D.  Feasibility  of  program  or  policy 

(10  points;  l=low,  10=high) 

Considerations: 

1.  Existence  of  infrastructure  to  support  implementation  and  sustainability  of  the 

program  or  policy  (e.g.,  medical  staff  and  facilities,  safety  staff  and  resources, 
manpower  availability) 

2.  Perceived  adequacy  of  funding  to  support  implementation  and  sustainability 

3.  Authority  to  implement  and  sustain  the  program  or  policy  is  held  or  obtainable  by 
the  implementing  organization(s) 

4.  Program  or  policy  will  not  undermine  essential  missions 

5.  Political  and  cultural  acceptability  of  program  or  policy 

6.  Accountability  and  responsibility  for  implementation  and  sustainability  exists  or  can 

be  established 

E.  Timeliness 

(5  points;  l=low,  5=high) 

Considerations: 

1.  Time  to  implementation 

2.  Time  to  results 

F.  Evaluation  of  program  or  policy 

(5  points;  l=low,  5=high) 

Considerations: 

1.  Ability  to  evaluate  effects  of  program  or  policy  (i.e.,  availability  of  metrics) 

2.  Benefits  of  program  or  policy  outweigh  the  costs  of  implementation  and 
sustainability 

3.  Collateral  benefits  as  a  result  of  implementation  (i.e.,  increased  readiness. 

decreased  attrition,  and  decreased  other  health  problems) 

quence  of  this  kind  of  thinking,  several  military  injury 
prevention  working  groups  have  adopted  a  quantitative 
approach  to  setting  priorities. The  criteria  em¬ 
ployed  by  these  military  working  groups  and  the  scoring 
of  criteria  are  provided  in  Table  4.  The  synthesis  of  crite¬ 
ria  for  prevention  priorities  enumerated  by  these  military 
working  groups  actually  began  at  the  CDC  in  2000  with 
work  started  there  by  one  of  the  authors  (BHJ). 


A  similar  set  of  criteria  to  those  for  setting  prevention 
priorities  can  be  used  to  set  research  priorities.  In  2002,  a 
preliminary  set  of  such  criteria  was  developed  by  the  U.S. 
Army  Center  for  Health  Promotion  and  Preventive  Med¬ 
icine  and  the  Johns  Hopkins  Center  for  Injury  Research 
and  Prevention  (Table  5).^^  In  setting  injury  prevention 
and  other  public  health  prevention  priorities,  a  primary 
criterion  is  scientific  evidence  that  effective  interventions 
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Table  5.  Suggested  priority-setting  criteria  for  military 
research 


A.  Program  or  policy  is  consistent  with  mission  of  the 
working  group/organization 

B.  Importance  of  problem  to  force  health  and  readiness 

Considerations: 

1.  Magnitude  and  severity  of  problem 

2.  High  costs  of  problem 

3.  Size  and/or  vulnerability  of  population  at  risk 

4.  Degree  of  concern  (command  or  public) 

5.  Gaps  in  knowledge  of  effective  prevention  strategies,  or 
modifiable  causes  and  risk  factors  exist 

6.  Military  uniqueness 

C.  Potential  value  of  research 

Considerations: 

1.  Cross-cutting  (cuts  across  types  of  injury) 

2.  Likelihood  of  identifying  discrete  modifiable  risk 
factors 

3.  Demonstrated  preventability  in  civilian  population 

D.  Feasibility  of  research  program  or  project 

Considerations: 

1.  Public  health  and  medical  infrastructure  exists  to 
support  research  efforts 

2.  Research  partners  exist 

3.  Technologic  feasibility  of  doing  research  (ability  to 
collect  data) 

4.  Adequacy  of  resources 


exist.  On  the  other  hand,  a  primary  criterion  for  setting 
research  priorities  is  that  adequate  evidence  of  effective 
interventions  does  not  exist.  Thus,  the  most  obvious  evi¬ 
dence  that  research  is  needed  is  when  a  big  problem  is 
identified,  but  no  research  is  found  to  support  preven¬ 
tion.  Insufficient  evidence  is  also  another  indicator  that 
research  is  needed,  as  multiple  studies  are  typically  re¬ 
quired  to  establish  the  validity  of  a  suggested  prevention 
strategy.  The  most  efficient  way  to  set  both  prevention 
and  research  priorities  may  be  to  conduct  both  at  the 
same  time,  because  the  process  for  identifying  important, 
preventable  injury  or  other  health  problems  will  be  the 
same,  and  the  criteria  for  setting  priorities  will  be  similar 
with  the  exception  that  there  will  be  evidence  to  support 
interventions  in  the  former  and  not  for  the  latter. 

Conclusion  and  Recommendations 

The  problem  of  injuries  for  the  military  is  large  enough  and 
costly  enough  to  warrant  the  time  and  resources  needed  to 
conduct  a  systematic,  data-driven,  and  evidence-based  pro¬ 
cess  of  defining  prevention  priorities.  To  effectively  im¬ 
plement  such  a  process,  all  the  functional  capabilities/ 
steps  of  the  public  health  approach  listed  in  Table  1  are 


necessary.  While  the  infrastructure  needed  to  address 
each  step  of  the  public  health  approach  exists  within  the 
Department  of  Defense,  it  also  needs  to  be  strengthened. 
Currently,  the  strongest  element  of  the  U.S.  military’s 
injury  prevention  system  is  medical  and  safety  surveil¬ 
lance.  While  improvements  such  as  the  addition  of  out¬ 
patient  injury  cause  coding  are  needed,  these  data  are 
currently  adequate  to  identify  significant  military  injury 
problems  and  to  monitor  changes  in  rates  of  injuries  over 
time  following  implementation  of  interventions,  pro¬ 
grams,  or  policies. 

In  regard  to  getting  prevention  information  to  those 
who  need  it,  the  infrastructure  for  disseminating  injury 
prevention  information  is  readily  available  through  the  mil¬ 
itary  service  safety  centers  and  chains  of  command.  Like¬ 
wise,  once  the  effectiveness  and  feasibility  of  an  injury 
prevention  strategy  has  been  demonstrated,  the  infra¬ 
structure  and  mechanisms  exist  within  the  military  to 
rapidly  implement  the  strategy. 

The  ability  to  evaluate  programs  and  document  suc¬ 
cess  at  the  installation  and  service  level  has  also  been 
demonstrated,^^’^^’^^  but  human  and  fiscal  resources  for 
this  essential  public  health  service  are  currently  limited. 
In  addition,  despite  the  fact  that  an  evidence-based  ap¬ 
proach  to  making  recommendations  for  injury  preven¬ 
tion  and  setting  priorities  has  been  employed  in  the  past, 
the  process  has  not  been  institutionalized  in  the 
military. 

Finally,  the  weakest  step  in  the  process  for  the  De¬ 
partment  of  Defense  is  research.  Even  though  injuries 
are  the  single  biggest  health  problem  of  all  of  the  mili¬ 
tary  services,  there  is  no  specific  injury-prevention 
scientific  or  technical  objective  to  which  resources  can 
be  routinely  applied,  with  the  exception  of  occasional 
monies  directed  to  address  ad  hoc  problems,  such  as 
traumatic  brain  injuries.  Without  an  injury-research 
program  guided  by  data-driven  priorities  and  without 
identification  of  gaps  in  prevention  knowledge, 
progress  with  military  injury  prevention  will  cease 
once  off-the-shelf  solutions  have  been  exhausted. 

With  the  above  considerations  in  mind,  the  follow¬ 
ing  recommendations  are  made  to  establish  a  compre¬ 
hensive,  evidence-based  approach  to  military  injury 
prevention: 

•  Use  readily  available  military  surveillance  databases 
(deaths,  disabilities,  hospitalization,  outpatient,  and 
safety)  to  identify  the  largest  and  most  severe  military 
injury  problems."^’ 

•  Commission  systematic  reviews  of  prevention  and 
safety  literature  to  determine  what  has  been  shown  to 
work  for  prevention  of  the  largest,  most  serious  mili¬ 
tary  injury  problems. 
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•  Establish  committees  of  medical  and  safety  subject 
matter  experts  to  routinely  access  and  set  priorities  for 
both  injury  prevention  research  and  program/policy 
implementation. 

•  Implement  or  adapt  proven  prevention  strategies  in  a 
prioritized  manner. 

•  Evaluate  effectiveness  of  all  implemented  policies,  pro¬ 
grams,  and  interventions/countermeasures. 

•  For  the  largest,  most  serious  injury  problems,  where 
information  on  effectiveness  does  not  exist  or  is  insuf¬ 
ficient  to  make  evidence-based  recommendations,  em¬ 
panel  subject  matter  experts  to  make  best-practice 
recommendations. 

•  For  the  largest,  most  serious  injury  problems  for  which 
evidence  for  prevention  does  not  exist  or  is  inadequate, 
commission  research. 

•  Establish  rouhne  channels  for  disseminahng  information 
from  each  of  the  steps  of  the  pubhc  health  and  evidence- 
based  decision-making  processes  to  ensure  that  key  stake¬ 
holders  receive  the  informahon  and  training  necessary  to 
effectively  reduce  the  impact  of  injuries  on  the  health  and 
readiness  of  military  personnel. 

•  Work  with  partners  in  injury  prevention  (e.g.,  the 
CDC,  CDC  Injury  Research  Centers,  NIH,  Agency  for 
Healthcare  Research  and  Quality)  to  complete  the  sys¬ 
tematic  reviews  and  research  needed  to  advance  injury 
prevention. 

In  following  these  recommendations,  the  military 
services  and  Department  of  Defense  have  an  opportu¬ 
nity  not  only  to  substantially  reduce  the  incidence  of 
injuries  to  service  members,  but  also  to  establish  a 
model  for  pubhc  health  practice  for  military  and  civil¬ 
ian  communities. 

The  papers  that  fohow  this  introduction  to  the  system¬ 
atic  evidence -based  process  of  injury  prevention  illus¬ 
trate:  (1)  how  priorities  can  be  set  using  military  surveil¬ 
lance  and  research  data;  (2)  how  surveillance  data  can  be 
used  to  define  and  monitor  injury  problems  for  the  mili¬ 
tary;  (3)  how  systematic  reviews  can  be  employed  to 
provide  military-relevant  information  on  what  works  to 
prevent  injuries;  and  (4)  what  have  been  some  of  the 
results  of  military  injury-prevention  program  evalua¬ 
tions  and  research. 


The  views  expressed  herein  are  the  views  of  the  author(s) 
and  do  not  reflect  the  official  policy  of  the  Department  of 
the  Army,  the  Department  of  Defense,  or  the  U.S. 
Government. 
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